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ABSTRACT 


An algorithm based on simple heuristics is presented for 
an important class of all-binary integer linear programs 
known as the set covering problem. In .Spite Of si bsmmer y 
special form, the set covering problem has many practical 
applications. Optimal solutions to problems derived fron 
these applications are difficuit to obtain using known 
metnods. Various Solution techniyues are investigated tased 
on heuristic algorithms that cbtain upper and lower bounds 


on the oftimal solution value together with branch and round 


enumeration. These solution technigues are effective on 
some problems. Cemputational results are reported for 
several large-scale real-world problems and several 
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A. iINTECDUCTION 


Set Covering Proklems (SCPs) comprise an important class 
Samiti! Uinawyee(O tjeeintege, Linear Programs (ILPs). The SCP 
model is well-known and has gany practical applications in 
diverse areas such as vehicle routing, facility iocation and 
capital budgeting. The set covering problem is a theoreti- 
Cae ete venlt problem in that —2t 2S Ne-complete [Ref. 1}. 
However, there exist several methods for obtaining solutions 
to SCPs for guite large real-worid problems. In this study, 
heuristics together with branch and bound enumeration are 
tested as a solution rethod for solving several large-scale 
Ser Ss. 

There are several reasons for uSiny heuristics witk 
kranch and bound instead of uSing cutting plane methods, 
LP-kased Eranch and Ecund, or scme other technique. PUES, 
not all researchers nave access to jood large-scale LP 
systems on which to Ease cutting plane or branch and round 
aegorat hms Any ccppetent researcher should be akle to 
program a heuristic-kased method with a modest ancunt of 
er forte The second reason for wanting heuristic-based 
methods is that more complicated technigues are sukject to 
failure as a result of degeneracy, numerical instability and 
slowness. For instance, the systems based on solving the LP 
belaxditon,s DOLh cutting plane abd branch and bound, ifeaueile 
when the LPs are difficult to solve because of their size, 
or because cf basis structures which are hard to invert, or 


because the LP gives weak bounds. See [Ref. 2] and [Ref. 3]. 


B. THE SED COVERGNG eR eLure 


The SCP is an integer program of the form: 


n 
(1) WEES, see, 
: ie 
Ze 
n 
Zz Soe a x 2b yn = 1 eee 
i X45 58y 7 Pi 
j= 
(3) ee oa na ry, j = 1,..o 
J 
where each a,,. = 0 or 1 


b > 9 and integer 


A minimal cost set cf columns must be selected from the 
coefficient matrix A SsUch that the right-hand sidcieuees 
covered or satisfied. Typically, right-hand-side values are 
all 1s. Closely related to the SCP is the set partitioning 
problem( SPP ) where (2) is replaced by (4). 
n 
(4) he Be fe = bi. ia —Oy ere), 
The SPP is the same as the SCP except that each row i must 


be covered exactly b. times instead of at least b, times. 
a aL 


C. USESFOr See AND essere 


Set covering probtlems and set partitioning problems have 
been studied widely because of their many practical applica- 
tions and simple binary structure. Bausch [{ Ref. 2] and 
Ealas and Padberg [Ref. 4] give a large collection of 
references to applications which are given below for 
completeness along with some more recent references. esos 


references 6,7,9,16 and 25 are unsighted. 
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Pee aCr i 

List Selection 

Tanker Routing 
Frequency Allocation 
Tracking Problems 
Vehicle Routing 

Sales Territory Design 
Colcring Problen 
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Capital Investment 
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Facilities Location 
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puapese Of 1) lustrating both the SCP 


example is described first. 


Corps 


supply using 7 possikle delivery routes. 


of the Republic of 
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here tor the 


and the SFP. 


headquarters of the First 
Seoryisiose 


It is assumed that 


the cost of each route is measured in dollars here, fut 
costs could also be measured in tine, mites travelleli, 
trucks used, etc. The incidence matrix A which is shown in 
Table 1 Consists ot Is and Os SUcn Eade 
: _ {1 aif route j goes theougnedivyistonen 
a QO otherwise. 


TABLE 1 
Truck Routing Example 
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Point 
#7 
#2 
#3 
4 4 
#5 
#6 
27) 
# € 


Costs 


a ee, | 
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~ —- »-~ O COO ~ = — IX 
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A set cf truck routes of minimal cost is to be determined in 
such a way that at least one truck route should go through 
each supply point. 


This problem is an SCE 


where A ={a_._. } 
1) 
An 
oye me mite le Wl ee Ceiavel 
C eat] SP 10612 Gee oe: 


Variakle i has the value 1 if truck route j is in the 


eZ 


Minimum cost set of routes and it has value 9 otherwise. 
The oftimal solution is 
kee (1,70, pO py a ee 
with cbjective value 17. The solution of the above problem 
uSing branch and bound will be demonstrated in Chapter 2. 
SUPpOse, OUNtNCsceher hand, thatetrucks are picking up 


supolies and that the net cost of route j is given by 


c = cost o£ route j - value c£ suppiies at all points in 

J 

COMES a). Beth thiSeeost Structure, mo overcovering of anv 
row may be allowed and this problem becomes an SPP. The 


optimal solution for the SPP is 

eee pC, 0,207 71,70, 1 a 
With objective value 18. Since this SPP is a restricticn cf 
the previous SCP, it is to be expected that the optimal 
soluticn to the SPP will be no better than the optimal solu- 
Elon to the SCP. 


D. SOLUTION METHODS FOR THE SCP 


Cne method of solving a general ILP is by attempting 
PoOmeaectinecwstne Optimal intecer solution of the ILP ase an 
extreme foint of a convex polyhedron generated by the orig- 
inal linear constraints plus some additional constraints 
Caerede  Cilcta, ine CeChhique 1s applied to the SCP by first 


solving the linear relaxation: 


MIN cx 
O), Sa eek 2.5 
oe 0. 


Solve tnis relaxation. If the solution is integer, then the 


solution must be optimal. Otherwise, erpivemwa Valid Cit, 


13 


i.e., a linear constraint which 1s sat ie@vedwoy a) eee 
solutions to SCP, but which is violated Dy @rne cimsen aie. 
integer solution. Add this contraint to themerchlenmmand 
solve the new restricted problen. Continue solving the 
restricted linear programs and adding outs igeu)  Cernceeae 
integer solution is ottained or numericai difficulties force 


a halt te the process. 


Zeeeecanch andes ound 


ee as = == — see ee SS ee 


Branch and Ecund is an optimization technicue that 
uses a tree search enumeration approach to the solution of a 


general ILP: 


Min CX 
(6) Saji 2 
x 250 


x integer. 
Following Garfinkel [Ref. 41], denote the set of feasible 
Solutions Coma oy 
Sees | ses 2 ees 2 Wy x integer } 

Instead of attempting to solve directly over 5S, the set is 
successively divided into smaller sets which have the fprop- 
erty that any optimal solution must be in at least cne of 
the sets. This 1s called separation and aoueettene | use 
trated by an enumeration tree with its root node at the top 


of the tree and with restricted subproblems bkelow the root 


(See Figure 2.1). Each node of the enumeraticn tree 
corresponds to a subfroblem of (6). That isp 1oG@eek Sane 
problem 

(ay MIN cx, x is in = 


where 7 1S a subSet “of 5 


En au Diner yeeee a is S with additional constraints which 
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meee certain 8variablies tor 0 or 1. As the enumeration 
proceeds further down the enumeration tree, the suksets 
becowe progressively smaller until it finaliy becomes 
possitle to solve (7) exactly or at least to determine 
Whether Cr mot ait containS a potentially optimal solution. 
Subproklems are discarded or "fathomed" when (7) 1s solved 
or when it is determined that a subset cannot contain a 
solution better than the best known solution to (6). Upper 
and lower tounds on the optimal soiution are calculated for 
each sSuklproblem allcwing for more efficient fathceming of 
nodes. The success or failure of branch and bound is 
largely dependent on the accuracy of these bounds. 

Pranch and bcund algorithms are often primal in the 
sense that they proceed from one feasible solutior to 
another until optimality is verified. In fact they may find 
optimal or near optimal solutions at an early stage in the 
enumeration process and spend the majority of the time veri- 


fying optimality by improving bounds. 


Ee LMPLESENTATECN 


fee Introd 16t10n 


Most large-scale mathematical programming frotlems 
have special structure which is exploited in the implementa- 
tion of mathematical frogramming solvers. Examples of such 
speciai structure are sparsity of the constraint matrix and 
the frequent occurrence of some coefficient values. To take 
advantage of this structure, the computer programs written 
for this study are written as subroutines embedded in a 


large-scale optimization test bed called the xX System or 


Simply xS [Ref. 42). XS is designed to solve linear 
programming problems, 0-1 programming problems, ronlinear 
programing problens and mixed O-1/linear /nonlinear 
programming problems. XS useS sparse matrix techniques 


Le: 


common to many mathematical programming systems. A mere 
specialized system using binary vectors to represen Ege. 
matrix [Ref. 43] might be faster for some problems but less 


flexible. 
2. input Data Format 


ih tise seuaw, to make data manipulation easy and 
convenient, the data format described by Bausch [Ref. 2} is 
used since this format has many advantages for large-scale 
problems. The advantages are as follows. 
a. Lt PS %eCom pace. 
b. Storage requirements are easily calculated. 
c. Data generaticn problems are simplified. 
d. Cclumn manipulation of data input is made easy 
Since all information for each column is contiguous. 
e. This column format is eaSily generated by ccmner- 
Cially availatle (MPS) problem gjeneration systems. 
The data input fcrmat consists of three sets of card images: 


a. Proklem dimensions. Format (316) (One Card) 


sl = Number of rows 
N = Number of columns 
NZEL = Number of non-zero elements. 
DC. Constbaint ranges. Format (2A 4¢3s022 Ges) ees) 
IR = Row index i 
RL = Lower range limit oF 
RU = Upper range limit (always @). 
c¢.- Column Data. (N or More Cards) “(2A55F 14 peo 5) 
JC = Column index j 
c = Column cost coefficient c. 
NCE = Numker of non-zero elements in the column 


IR crow addresses orf non-zero coefficients. 
If NCE is greater than 9, additional column cards 


are needed to hold the row addresses for that 
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COMIN. “ne LOENau tos additional column cards is 
(ZOxX, VOI5) < 
An example of this data format is shown in Appendix A for 


wre tEWVeck LOouting exarple of Table 1. 
3- Test Problems 


Eight test problems are evaluated in this thesis. 
These Froblems consist of four real-world proklems 
(ieEdcan,wnus, Tiger dnd Truck), and four artificial rrob- 
lems (Steinerl, Steiner2, Steinria and Steinr2a). Steiner] 
and Steiner2 are problems devised in [Ref. 49] ana are guar- 
anteed to require extensive enumeration when using LP-kased 
Eranch and bound since the LP bounds are so weak. Steinrila 
and Steinr2a are Steiner? and Steiner2 transposed, respec- 
PEyely. 

Some of the problems are, in fact, pure SPPs. 
However, we have converted these problems into SCPs 
reasoning that the derived SCP should still be representa- 
pave fon .a true SCP. The characteristics of these proktlems 
are shown in Table 2 where NZEL is the total number o£ 
hnonzeros in the constraint matrix and NCE is the average 
humber cf nonzeros in each column. All these proklems 
except Truck are typical set covering problems which have 
right-hand sides egual to 1. The Truck problem has a 
general right-hand side. All of these test problems were 
Erion an ©EBhe0s3 under VM/CMS. Computation times reported 
in the following chapters are accurate to the number of 


decimal flaces shown. 


ig 


eS SS ES A AS SS A A SS ES A a SS LS LS 


TABLE 2 

| Froblem Dimensions 

| Problem Rows Columns NZEL NCE Model 
| American 5 28 Is. 57.293 6.0 SPP 
| Bus 56 530 3539 6.3 SPP 
| Steiner 4! PLA 352 1230 SCP 
| SteinerlA 27 10h So 1320 SCP 
Steiner2 3.30 G5 9917 22 SCP 
| Steiner2a 45 330 991 Bone 0 SCP 
Tiger 160 636 4134 oe SPD 
| TEU Zo 4752 30075 Gre 0 SCP 
| ee — a a | 
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A. iINIRCDUCTION 


in this chapter we introduce "branch and bound" enuzera- 
tion which will be used in chapter 5 to solve SCPs. Branch 
and bound iS an optimization techniyue that uses tree 
enumeration together with urper bounds and lower bcunds on 
the objective functicn. These bounds heip to accelerate the 
fathominyg process and reduce enumeration. In this chapter, 
we descrite branch and bound in terms of a@ Minimizing binary 
TIP, and discuss tke importance of good bounds and good 
Eranching strategies. ind pimeaGy f£bP, a Separaticn is 
effected Ey fixing a binary variable to its possible values, 
Ueandeieee hus, every Separation of a problem is, in fact, a 
partition of the prorlem into two subproblenms. 

the discussion of branch and bound is limited to a 
"depth-first" search or exploration of the enumeration tree 
Since this is the method that was used in this research. 


More general technigues are possible ( See Garfinkel an‘4 


Nemhauser [kef. 43]. ) but these all require substantially 
more storage and general overhead. Most commercial Eranch 
and bound packages utilize a depth=l1lrst search. 


Cepth-first search simply means that when a separation is 
defined, one of the nodes created by the separation is imme- 
diately selected to re the next subproblem, and when a node 
is fathomed, the enumeration always backtracks to the nost 
recently created live node which is the “twin of a node 
already explored. 

PHeSceabe NtwO MUPOortant Sarts of any branch and LEound 
algoritho. er Site, good upper and lower bounds must be 


obtained on the optimal solution. The closer the bounds are 


ue, 


to the optimal solution, the tewer noges (Sub pro Sten) ian ie 
be enumerated. There exist several ways of obtaining upper 
and Lower bounds on an SCP. Methods for obtaining comrputa- 
tionally simple lower bounds will be described in Chapter 4. 
Upper bounds on the optimal solution are given hy Simeurisci. 
caliy obtained feasitle solutions to the MILP. Yeuristic 
solution methods for the SCP will be described in detail in 
the next chapter. 

The second important part of the Dranch and Pound jaacc 
rithm is the method of determining which variable tc f1x 
first when a separation is defined at a node in the tree ani 
whether that variable should be fixed to 0 or 1. by gtal 
selection process is caved a “brane hing Strategy vee 
Branching strategies are derendent on the methods being used 
for obtaining upper and lower bounds and the actual strat- 
egies to be used will be discussed in chapter 5. It seems 
obvious that if a good guess can be made aS to which vari- 
ables must Fe in the optimal solution, then fixing one of 
those variables first to Tf would be a good tranching 
strategy. O£ course, guesSing is very diificult; other as— 
we would have guessed the solution to the whole problen. 
Another likely strategy may be to select the least attrac- 
tive variable in the incumbent and set that variable to 0. 


Unfortunately, as will be seen in chapnter 5, no Single rule 


-~* 


seems to work well on all problems anda certain amount o= 


case-|Ly-case experimentation is necessary. 


B. FATHOMING CRITERICN 


To accelerate the enumeration process and save computing 
time we need a criterion to decide whether or net a 
subproblem should be discarded at a certain point of the 
algorithr. Suppose that several steps of the enumeration 


have already been ferformed and that a subproblem at a 


20 


particular node in the tree is being considered. eee = 
denote the smailest feasible objective value found thus far 
in the enumeration. Clearly, BEST iS an upper bound on the 
Optimal Solution to the TP. The feasibie solution 
SOEreCSPONndInG EOUBES1 isewcalled the “incumbent.™ 

Now, let CLBND denote a lower bound on the optimal solu- 
tion to the ILP given the restrictions at the current node. 
Seene elo acrined towne sanctinity 12 ao. feasible soluticn to 
the ILP can be found given the current restrictions. Let 
CUBND denote a upper bound on the optimal objective value 
corresponding to a feasible solution to the ILP given the 
current restrictions. If BEST > CUBND, let BEST = CUBND and 
let the corresponding solution be the new incumbent. Now, 
the efficiency of fEranch and bound enumeration is kased on 
the fact that explicit enumeration need not be extended 


Lelow the current node if the "fathoming criterion" is met: 
PIenoOmiIng ecrieericn: CLBND 2 BEST. 


Pomepreblems with integer costs, fractional values for CLsNn 
should ke rounded up to the nearest integer. 

PEO ascOnpUutadrLlenal ss Viewpoint, at 2S useful to split 
the above test into two tests, however. First, compute 
CLBND and test if CLBND 2 BEST; tte tS the . node is 
fathomed. If not, only then compute CUBND, update BEST if 
Ser rOoprrare. and repeat the test. Thais avoids some 


unnecessary computaticn of upper kLounds. 


C. SEPARATION AND BRANCHING 


"Branching" describes the process whereby an unexplored 
subproblem is selected for expioration, 1.e., upper and 
lower bounds are ccmputed for the node ani the node is 
either fathomed or separated. uSepabatiomtois the orocess 


wherekty the current subproblem is separated into two or more 


2a 


subsukproblems, at least one of Which mG2e) cograii ae 
optimal solution to the current subproblem if such a solu- 
tion €EXiSts. 

In a binary LP using depth-first Search, branching ard 
Separation are intertwined. A separation is always a 
partition based on fixing a specific variable to 0 of to 1. 
After a Separation cne of the live nodes just created nust 
be immediately selected for branching. If a node is 
fathomed, the most recently created live node must be 


selected for exploration. 


D. ALGORITHM AND EXA¥FPLE 


i Algorithm 


The following branch and bound algorithm uses depth- 
first exploration of the enumeration tree. The. legac mirs 


exactly that used in the programs written for this thesis. 


Algorithm: Depth-First Branch and Bound 

STEP O. (Ingtializacron) 
Let BEST = Oo; saenm=—a. 

STEP 1. Compute CUBND given restrictions defined eby 
STAC 
J£ CLEND > BEST. qomterceepm or 

STEP 2. Compute CUBND given restrictions defined Ey 
SLACK. 
If CUBND < BEST, then let BEST = CUBND and save 
incumbent. | 
Tf CLBEND 2 BEST, go tonctcomsr 

STEP 3. (S8ranching) Select an unfixed variable j to fix 
and determine whether to fix it to 0 or 1. 

OIEP 4. Put vertex j in SITACK “with jintornagtuonmnames— 
ting whether it is fixed to 0 or 1 and that 
its twin has not yet been explored. 


Goto ster ie 
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ames. (Backtrack) 
If STACK = 9, then go to step 7. 

Sie o.  PcohOve=g ELOne top Of STACK. 
tewats tye hbase been explored, go to ster 5. 
Cele nvSse EDULE Uden Gh  osrnhCK s1xing j to the 
complemert of its previous value and noting that 
its twin has already been explored. 
GOerTO Steer, 1. 


TELMInNat Len 


72) 
A 
tr 
Ao) 
~) 
e 


If BEST =®, there exists no feasibie solution. 
Otherwise, current incumbent is optinal. 


End of Algorithm: Branch and Bound 
2. Example 


The example relow illustrates the above algorithr on 
the SCP defined in Tatle 1. Lower bounds on the solutions 
at each node are obtained by finding a feasibie soluticn to 
the dual of the LP relaxation of the SCP (See section EF in 
Gaapter 4.). The upper bounds are obtained by using an 
Naddition heuristic" which successively adds columns toa 
partial cover until a complete cover is obktained (See 
Sectercn Gein Chaprer 35). Separation is effected by 


randomly selecting a variable x. amony all variables not 
J 


ime ewec (ecole SOUuleeneOntalned by the addition heuristic. 


Dicmettatechmeconecsponding tO x > 1 1s explored first. 
The enumeration tree iS Shown in Figure 2.1. 


as inatvealize: REST =® 
Doeode-0- CLEND 1730 
CLBND < BEST so continue. 
CUBND sieaehe a =D OO ga Oy 1.) 


CUBND < BEST so let BEST = CUBND and Se 


becomes the incumbent. 


Ze 


Ce 


de 


Laer 


tee 


Node 1: 


Nede 2: 


Nede 3: 


Node 4: 


CLBND < BEST so variable a is selected 
for branching. 1x )vWdaetaone 2 EO IMEtEse. 
Given that he = 1, CLBEND = 2520. 


Since CLBND 2 BEST, backteaek sto ene ewe 
of this node which has not been explored. 
Given that cg 0, ClLakD = Igaoe 


CLBEND < BEST so GContinue. 
CUBND = 18.0. No improvement over incumtent. 


Since CLBND < BES2, selcau “eS Eor branching. 


Set x = 1. 
5 


Given that re O and ie 1} CLBEND = eo 


Since CLBND 2 BFST, backtrack to the twin 
of this node which has not been explorede. 
Given that Oe 0 and x = 0, CLBEND = "toe 


Since CLeNp 4 BES1  Comerniie. 


CUBND = 17.0 for k= 1 40) eo OOO 


Since CUBND < BFST, let BEST = CUBND and 


let X be the new incumbent. 


Since CLBEND 2 BES?) bacweracw 
No live nodes exist, so the current incumbent 


a is optimal with objective value 17.0. 
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mm a a mae eee ee ee ee ee 


a7 .0 , 11620) 


es eee 


(pe, 18.0) 


x Q 





i 
10 


(18.05) 1850) GPO, 17.0) 


Note. (PalrTS anemneLEND, BEST) 


Ss > a ae 


(L.—_--; oo ee gee gee oe 2 Oe 


Pigure 2.1 Enumeration Tree for Truck Routing Example. 
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III. HEURISTIC SOLUTION TECHNIQUES 


A. INTRODUCTION 


Two kLasic heuristic techniques exist for obtaining good 
feasible solutions to SCPs:; “addition Sveupa2sties) and 
"deletion" heuristics. These two heuristics are used in 
this study for the purpose of generating solution sets and 
upper Foundcs. These heuristics are not guaranteed to solve 
the SCP optimally but can be used to get good upper bounds 
or the optimal solutions which are essential in the Eranch 
and bound enumeration. Feasible solutions to the SCP are 
easily oktained because of the SCP's greater than or equal 
to constraints and nonnegative constraint vey aims bs Ge 


Computational results are given in section D. 


B. ADDITION HEURISTIC 


An addition heuristic begins with the infeasible solu- 


tion x=0 and successively sets to 1 that variable x. which 
ae J 


myopicaliy minimizes effective cost. The effective ccst 


associated with x 18 c¢ /p , Where p) is a penalty which eee 
J J 


some way reflects the amounts of infeasibility currently 


being contributed by x = 0. The addition heuristic can be 


Stoop ed when a feasible solution is obtained but it is 
possible that the cover produced is not minimal and a second 
Phase should be added which deletes any columns in the cover 


Which rel redctindant. 
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Rigerithns: 


Addition Heuristic 


HROUtiweene SCP Nama x ama vectors A, c and b. 


Cucout ; 
Sy ee 0. 


SUR 2 Ie 


SEP 2 


Upper bound to SCP solution. 


Wine eiaezatLon 
io — eo ee eee = Bel, 2, . 1}; 
=e bye CUaND = 0. 


For each column jéJ 


Compute a penalty p > U- 
J 


Let h. = number of nonzeros in column j. 
J 

iO sworn edo eelmny, go. tO Step age 

let 


Cc ° 


Otherwise, 


——_— 


Ee 
y 


i = argnmin 
Jo a 


PB iis —_ Sie 4 5 


0 
CUAND + Jee, . 
Jo 
FOr@each 1 Such that a. = 1 


1 
Jo 


CUBND = 


ete tO ore 
1 


1 


Tf b' = 0 (update column sums) 
a 


For each j such that a. =a. = 1, 
a5 ve 

bets a= ho — 71. 
J J 

Tf h = 
J 

eae “O7eGgo £0 STEP 3. 


Otherwise, 


Ope ete = a= 3% 


For each column jéJ 


Update penalties p if necessary. 
5 


Go to ster 1. 


PS | 


STEP 3. "Generating minimai cover" 
For each eack jé J! 
If celumn 3 is pedumdane 
Let GUBND = CUBND Reece. 


Leta? = 97 —ay- 
Sine ut. “Termination 
Halt. ° If J* is a cover, CUBND 2s aneie eee eom a 
on the SCF. 
Otherwise, no feasibie solution exists. 


End of Addition Heuristic 


Two different penalty functions have been tested with 


the above addition heuristic: p,= h,.andp,= k,. where k. 
J a 1 J 


is the initial column sum (number of nonzeros in the coiumn’ 


which is never updated. Kovac [Ref. 44] suggests using k. 


as part of a heuristic for obtaining both upper and lower 
bounds on the optimal solution to an SCP. The results 
obtained using this penalty are not reported here, however, 
Since they are so foor. It should be noted that tore 
complicated penalty functzons could Certavmiy e-em? 


such as p = log(h.). In addition, instead of selecting the 
3) J 

Minimum c /p. the minimum of a more general functional fcorn 
a 


g(c _,p.) could be selected. 
i ow) 


C. DELETION HEURISTIC 


A deietion heuristic begins with the feasible solution 


x=? and successively sets to 0 that variable x which 
oe J 


myOpically minimizes effective profit c /p . Here er is some 
J 


a 
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Eowartyebert.ccfting Ene amounm: Of OVercCOVering which column j 
VomrcoOwenl puting. The deletion heuristic stops when no 
aiditional variables can be set to QO without creating an 
infeasiblility implying that the cover obtained is fininal. 


The tellowing algorithm Carries out the above ideas. 


Algorithm: Deletion Heuristic 
input: The SCP matrix and vectors A, c and b. 
Gic Slt Upper bound to SCP. 
SiE2 0s ert talazation"™ 
=o aos, hm. 1.2, ee), 
n 
3° = Vpeeete= b, CUBND = >) c. 
Ri 
For each i€I, 


let h.= number of nonzeros in row i. 
at 


For each JEJ, | 


let h = number of nonzeros in column j. 
7 : 
compute a penalty p> 0. 
J 


Selo. OeceeJ ts empty; jo to STEP 4. 


Cc e 
jo. = argmax = 
0 jéJ jen 
J 
Slee ormecacnm. SUCH that a = 1, 
13 
0) 


ite s= h 7 go to STEP 4. 
a 1 


STEP 82 J = J - jj 


0 
CUSHD = CUBND = c. 
J 
For each row j} such that a. = 1. 
1) 
aS ee oe © ee I 


aS 


1 
Gorto SIEF t. 


Px) 


STEP 4. For each a = 1 let J' = Jt # Gj 
1) 
0 


For each a, = 1 let®h® -Sp 
Tag A al. 


if b = 0, update colunne=sUm iter eing 
+ J 
J= J - j for any h. = QO. 
J 


If b*' < 9, go to STEP 6. 
STEP 5. For each column Gear 


Update penalties p. if necessary. 


GO “tors lit wa 
STEP 6. “TeELNina tien 
Halt. If J' is cover, CUBND is an upper bcund 
on the SCF. 
Otherwise, no feasible solution exists. 


End of Algorithm Deletion Heuristic 


The deletion heuristic has only Been Gested=us ince 
J.” 


The comments on the fossible use of more geheral functional 
forms in the addition heuristic are also valid here, but 


have not keen tested. 


D. COMPUTATIONAL RESULTS 


An addition heuristic and a deletion heuristic have heen 
coded for purposes cf comparison. The resultS are summa- 
rized in Table 3. As can be seen, the addition heuristic is 
faster than deletion heuristic, but the upper bound from the 
addition heuristic is not as good as that oktained by the 
deletion heuristic except for problems Steiner1A and Truck. 
Although these results tend to indicate that the deletion 
heuristic is better, the true test of usefulness in solving 
SCPs will have to wait until chapter 5 where the heuristics 


are embedded in a branch and bound algorithn. 


3.) 


- 
| 
Alas 3 | 
’ | 
Computational Results of Upper Bounds 

| 

| 


| 

| 

| 

| 

| Prebiem Addition Weterroa 

| Value zive Value Time 

| American BIE 72 Oies'5 3e565 35513 | 
| Bus 5 eZee 0203 ye || oe 0.64 
| Steinert es; 0 0.00 19.0 O00 | 
| Steirria 9.0 Cau G0 0.00 | 
| Steinerz 3220 0.00 B02.0 SO | 
| S femenr Za \6-30 0.00 16.0 0.24 | 
| ay ohefspe 5 Io 4 0.11 5973 0.92 

7 wc k 3697 .€48 1.19 389 #62 Soo 35 
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The above heuristic techniques are one-pass methods and 
are the cnly methcds implemented in this research. 
Multiple-pass methods exist and are mentioned here for 
completeness. The most straightforward multiple-pass method 
is) called the "i=-opt" method fRef. 51]. Tit setic th Od srl Est 
uses an addition or deletion heuristic to obtain a mininal 
cover. Then, each column in the current solution is checked? 
@iateseerhe COLUMNS NOt im the solution to determine ifa 


one-for-one exchange can be made which maintains a feasible 


cover while reducing total cost. The i-opt method is an 
example ci an “exchange heuristic." “fhe basic idea can be 
extended to a kK-~way exchange resuiting in the "k-opt" 
method. 
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A. INTRCDUCTION 


Several methods cf finding lower bounds for solutions to 
the SCP are described in this chapter. Getting gocd lower 
bounds on the optimal solution to an SCP) is @emaiea 
optimal solutions are to be okftained using Eranch and bound. 
Lower bounds are also necessary if the accuracy of nonof- 
timal sclutions is te be bounded when branch and bound fails 
to find the optimal solution. There are many possitle 
methods of obtaining lower bounds on the SCP, all tased on 
solving some relaxation of the the associated ILP. Several 
methods have been coded and are compared to decide which 
method should be employed within the branch ard bound 
enumeration. Although these bounds have not been used for 
solving any SPPs, it should be noted that they are all valid 
for the SPP since the SCP is a relaxation of the SPP. 

A feasible solution to the dual of the LP relaxation of 
the SCP provides one easily obtainable lower bound. Also, a 
column partitioning method is given in which the SCF is 
partitioned into small SCPs which can be solved exactly and 
whose soiution values may be summed to give a lower bound. 
Another lower bound which is tested is the sum of all of the 
Minimal row covering fractions. All these methods are coded 


and computational results compared in section E. 


B. DUAL LP RELAXATICN LOWER BOUND 


One cbvious relaxation of the SCP which can be solved to 
obtain a lower bound is the linear programming (LP) 
relaxation. This technique has been used successfully in 


Many cases [Ref. 45]. But a problem with the LP lower bound 


a2 


[ost vdtwleebay De GuUrficuiteto Solve the LPs associated with 
Hang ees. (See for example Bausch [ Ref. 2] and Saikin and 
Koncal {Ref. 3]. ) This is true because the LP may be auite 
large, highly degenerate and have a basis structure which is 
numerically unstable. However, it is possible to geta 
guick lower bound on the SCP by just finding a feasible 


soluticn to the dual of the LP relaxation of the SCP Since, 
“We lew * (IP ) = V®{LP }) < v* (SC2) 
D D Pp 
where ee) = objective value for a feasible solution 


tc the dual of the LP relaxation, 


ee = optimal value of the dual LP relaxatiocn, 


eee optimal value of primal LP relaxaticr, 


and 


ve (SGP) 


Peer CciOLC (new ceetOL Olwenesse the dual toe the LP 


cptimal value of the SCP. 


relaxation of the SCF is 


rr 


max bu- iy 
a 
See le Lv < C 
Ree ay! = 0. 


The dual LP looks very Similar to the SCP itself if the dual 
Vatiatles vare set to zero, ido hp Lome tno d ay rOr 
obtaining a feasible solution can be devised in a way that 


is Ssitilar to the greedy addition heuristic for the SCP. 


Algorithm: Dual LP Relaxation Lower Bound (DLPRLB) 
Hip iae seh coc trIcYent Nattix and vectors A, c and fC. 
Cwm wower bound CLBND to the SCP 
Speen Literal Za tion" 
CLBND = QO 
ee eee, , My eeeetie— 1,2 ps «6 7 1} 


POmecach tél tet. h 
it 


Number of nonzeros 
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1 we Ow 1. 


SIEP 1. If I is empty, go to SPER ase ee 


b 
. i 
1 = argna 
0 1€1 h. 
iL 
f= 1 ee 
0 
Sls? 2. ce rin fe 
j j42) 1 
0 al 
0 
OF ea 
Jo 
STEP 3. CLBEND = CiBND + tee. 
ce 
0 
STEP 4. For each row 1 Su@a thatua sa 
1) 
0 
For each column j such that a, = 1, 
1) 
Update cost coefficients 
C.(= (Canc ee 
; J J Jy 


Fepecat ol bee 
SiEPeo. Her Ni hat Jone 
Halt. CLEND is a lower bound on the SCP. 
End of Algorithm DLPRIB 
At each iteration of the algorithmn,, dual variable Bi, 
would @oe set to uc The actwel vaisies of etnemaua 
Jo 
variables need not be retained, however, since the value of 
the dual objective function is just blu. 
Hey [Ref. 45] gives a somewhat more complicated method 
for finding a feasible soluticn to the dual relaxation of 
the SCP and this is tested along with the method descrited 


above. 
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C. PARTITIONING LOWEE BOUND 


Marsten [Ref-. 50] gives a method for finding a lower 
bound on the SCP by solving subproblems of the SCP defined 
OMecertain —PpakEtitions Gf the coefficient matrix. ie oh 
easier te describe the method, however, in terms of a maxin- 
Peat icy. croclen.. 

Prown, McBride and Wood [Ref. 46] give a way to calcu- 
late an upper bound using a partition of the coiumns for a 
problem of the forn: 

MAX cx 

S-T. Ax S$ b 

x binary. 
They use the bound fcr estimating the size of the maxinun 
embedded generalized network in an LP constraint matrix 
where ¢c =i, b= 2, and A is the transpose of the 0-1 inci- 
Gence matrix associated with an LP constraint matrix. Their 
bound is found as foilows. 


Let ae and Bs Nesdmeartltlonme or thevcolumns of A and let z, 
= and ces beumbiecmmsclutTonmor, (the Maximazation proclem on 
Ny and oe respectively. Then, z ar - a 


Sete - is intelligently chosen, 2; can be computed exactly. 


Then, A Lecomes ap and the partitioning scheme 1S recur- 
Sively repeated untii e can be easily solved also. 


The SCP can be cenverted to a maximization problem by 
substituting variables 1-y for x and multiplyiny the okjec- 


Live: fineuion by — 1. Hits eteas jOt nard to see that a 


lower bound on the SCP can be oktained using the above 


method. Of course, the method can be applied directly 
without making the substitution. The partition of the 
columns 

> is created with respect to an arbitrary row i. The 
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columns of Aare those columns Yoreieo C€mtadin ing 


nonzeros. The minimum cost among those cost coefficients 
contributes an additive term to the lower bound. The vari- 
ables included in the partition are never considered again 
and all rows with nonzero intersections in the partition are 


also never considered again. 


Algorithn: Partitioning Lower Bound 
Input; SCP coefficient Matrix and Veetorece c and E. 
Cutrbrut: Lower bound CLBND to the SCP 
SipP 0... -Wknaner alla acea on | 
I = {1,2,5..,0}, «= {1725 eee) 
For each i€I, 


let h.= number cf ‘nonzeros in row i 
a 


STEP 1. If 1 aS eEnpty, go) toustepaue 


« 


Let i. = argmin h 
0 ea 


a 
f= Soe 
9) 
SIFP 2. (Pind thesgb) Mininun C (ee ron Aa 
a 
0 
Let j = - arg Min oC ps 
0 730° = ja 
a 
02 
Let CLBND = CLBND tc. 
J 
0 
Let c =& 
3 
0 
Let 95.0) =) cells 
i a 
0 0 
Ti b 30, go tossten sz. 
1 
0 
SIEP 3. Fort each 9) sich we hanna — se 
i 3 
0 
For each 1 such that a = 4 
nies Be 


SG 


sO "to Step. 1. 
SizP aun. er pination 
Pouce Ny ers lower boUnd™to the SCP. 


End of Algorithm Partitioning Lower Bound 


D. KOVAC'S LOWER BOUND 


Consider the basic model of the SCP with all right-hand- 


Side values equal to 1. 


n 
(8) MIN Cpl xe 
~ 


Bi 
n 

Seal a x 2 1 mS | Ae od aia 
2 935 %5 afi 
j= 1 
6) Iealsovevien iets | jvetseea, Tic 


The collowing lemma [Ref. 44] provides a lower bcund for 
the akove SCP. 


te 
Lemma Denote the optimum of problem (8) by 2Z 
then 


nN 
* 
iS) on eee, =F 
Al 


1=1 
where £ is the minimal covering fraction of row i: 
at 
0 e 
et ee Osc) <= fy a = 1}. 
aL 5 ij 


(Proof) Define the fcllowing new problen. 


Bo 


(10) MIND DO cry 


jk 
j=1 k=1 ae at 
n m 
So 2 pee carer 
du 455k jk 
j=1 k=1 
y. binary So ly eee ees Vesey 0 
ak 
where Bo cee) = 1 
eee 13 
13k 
0 otherwise. 


To any feasible selution x Of EhewSer ecu corresponds 
a solution of problem (10) 1n such a Way | 2 hae objective 
function values are equal. Specifically, for each 2a 
J 
= = | forvalia< On the other hand the minimum of probien 
3 
(10) is obviously F. This proves the statement (9) of the 
lemma. 
It is easy to extend the Kovac bound to problems with 
general right-hand-sides Ly using 
m 
F = eet oe 
»» a 
1=1 


E. COMPUTATIONAL RESULTS 


Computational results for the lower bounds described 
above are reported here for all the test problems (See Takle 
u. “DNR* indicates did met erin It 1s not clear that the 
tightest lower bound for the complete problem will perform 
the best in the rranch and bound enumeration, but some posi- 
tive correlation is to be expected. The computational speed 
of the bound is also a consideration in branch and Eound and 
some increase in speed may be traded for a loss in accuracy. 


Thus, the results given here are a guide to which bound wri 
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Pemi@sae ClLicctive in the branch and bound algorithm Etut only 
testing with that aigorithm can determine true effective- 
ness. The dwal LP @relaxat@on method (DLPRLB) appears to be 
the best in the problems Bus, Steinerl, Tiger and American. 
Petilaliy, eset) DEPRLEpdnd Hey's method do outstandingly well 
om SUS. Kovac's heuristic appears to be superior to the 
Gener bounds in Truck, Steinrla, Steiner2 and SteilnrZ2a. FOr 
Steinrla and Steinr2a, the bounds are tight indicating that 
Pom OCd NedersliG wSOlULLONS Can De obtained, the branch and 


bound aigorithm should terminate very quickly. 


3 


ere em ee | ee re eee ee se ee Se ee 


| 
| 


Problem 
American 
EUs 
Steiner 
Steinrila 
Steiner2 
Steinr2a 
Tiger 


Truck 


a me ee rr er re ee Se es ee ee eee 


V5 0 


Sy ae a 


? 


TABLE 4 


Computational Results of Lower Bounds 


rea 
14.7 
iS ene) 


Se 
ZUIG= 


Partitioning 
Value Time 
1227 OG 
2.99 0.00 
G0e) LOO 
DOO 00 
1 ORO 2 0G 
Doe 0%.00 
30.29 9.00 
149.1 0.00 
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VomcomMreulcAteonnl EXPERIENCE AND DIFFICULTIES 


All of the computational results reported in this 
secticn are for SCPs. The algorithms performed very well on 
some of the problems Eut not on the others. At first, small 
test problems were used to check the correctness of the 
algorithms. All algorithms worked weil on these small prob- 
iems. The algorithms were then tested on the eight frecklems 
described in chapter 1. To soive these problems, we have 
used five methods to see which method is more effective than 


tne others. The descriptions of tne methods follow. 


Wepnod) 1; Lower beund: Dual Le relaxation. 
Upper bound: Addition heuristic. 
Separation: loo artgmin c /k. among all 
a J 


Wri cil bemt Soli ti. om 
J 


Braneming: x = 1 first. 
J 
0 
Method 2: Lower Ecund: Dual LP relaxation. 
Upper round: Addition heuristic. 


Separation: 5) = argndaxec /h, among all 
ee) 


Foto Current Solution. 


Beanech ange. x. = 90 first. 
J 


Method 3: Lower bcund: Dual LP relaxation. 
Upver Lrcund: Deletion heuristic. 


Separation: argmin c /h. among all variables 
aes, 
j} not in current solution. 


BeancGhinhgo Xoe=— 1 Lirst. 
J 
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Method 4: Lower Lound: Dual LP relaxation. 
Upper cound: Deletion heuristic. 


Separation: argmax c_ /h, among all variabies 
J J 


j not in CUErent 9solucaon, 


Branchang;: x =. 0 segs. 
5 


Method S: Lower Lound: Kovac's 
Upper Eound: Addition heurrerre: 


Separation: arcmin ¢ / Paloma alley atta beee 
Se: 


j not in CurEnent Seuuticnue 


BEANChING< 3x 9= 0 ipeet es ee 
J 
As illustrated in Table 5, three problems were not 
solved optimally. We denote the actual percentage with 


respect to the optimal value as a "ZOPT" and the provakly 
optimal percentage as "%POPT." %&%POPT denotes the arount by 
which were able to preve that the best solution found varied 
from the optimal solution without knowing the optimal solu- 
tion. This value is obtained by changing the CLBND 2 FEST 
tests in the branch and bound alyorithm to 

CLBND = BEST-EPS where EPS is. an allowable amount of error. 
If the branch and bound allgorithn then fakes. be £ close 
that the incumbent solution is within 100%° (BEST+EPS) /BEST 


of the optimal solwtien- 
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<<  e ee e nees ee ees ee e ee  n ee ee ee ee ee ee ee oes eee ee ee ee een eee eee 


TABLE 5 


Computational Results Compared with Previous Results 


New Results Previous Results 
Preblems OPT SZOPT &%POPT Method Time TOPT ILPtime 
American Ve 72E 1020 UNO? 1 0.09 100 20.55 
Bus 4.696 100 100 2 0.54 100 1.02 
Steiner1 18.0 100 100 4 30.98 100 25.13 
Steiner1A 9.0 100 100 5 0.01 100 0.91 
Steiner2 30.0 100 100 4 96.74 100 527.08 
Steiner2A 15.0 100 100 5 (geod 100 3s 1 
Tiger Sew > lee 0 ee 495.0 1 1.0 100 RY 
meie ki & fd 140.0 5 yon a 2 ? 


TE me tn EE TT TT A ET EE SS Ss AR cae lita 


ae 
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a a ee ee aes a ee a a pd ee 


All these problems except Truck abet) pica set (eoverun, 
problems which have right-hand-side vaiues equal to 1. The 
Truck froklem has the general riyht-hand-side form shown in 
Equattom (2 yee The ccmputational results are summarized in 
Table 5. These results are the best of the various solution 
methods tried. "Previous Results" indicate either those 
times reported by Bausch or the times we recorded using the 
methods cf Bausch. Kunning times on Steiner2 are for 10000 
nodes only; optimality was not proven in either our or 
Patsc ms COMPU cd Lon. Table 6 shows the comparison between 
the different soluticn technigues on each of the probiens. 
The problems marked with * were not solved optimally within 


1 minute of CPU time. 
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TABLE 6 


Comparison of Various Solution Methods 


| 
| 
| | 
| Prcblem OPT Method 1 Method 2 Method 3 Method 4 | 
Value Time Value Tine Value Time Value Time | 

American® 1.726 1.89 lah 12CSme Viet 1.89 12.10 1.89 Set 
| Pus 4.696 4.696 9.82 4.696 0.54 4.696 16.61 4.696 4.24 | 
| Steuner 1 ion 18.0 oan ioe Gemme 36m 25 289 a0 24.51 1B 30.92 | 
| Sternrla © 920 9.0 30.21 S20 We 32593 9.0 33. 83 9.0 39522 | 
Steiner2 30.0 230.0 Vda 69 050) 100589 = s040ue105. 11 Sasweo 96.74 | 
| Seine an 4520 15.0 154.53 15.0 144.58 Sem 26.07 lee 0 = Odo. o | 

Tiger* 52 oo 0-5 540 Poe So ae! we S428. "Soni? Avec 

| Truck* ? 354.5 19.91 350.7 15.61 389.6 56.42 389.1 SING 
I at sie Oe OM oe _ 
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One cf the results of the tests was that the deletion 
heuristic usually produces better feasible solutions than 
the addition heuristic both initially and furever eon 
tree. This leads to the enumeration of fewer nodes with the 
deletion heuristic. Unfortunately, it does not lead to 


faster times because the deletion  heuevstaiee. > Jooumenue. 


slower than the addition heuristic. For. example, “wWsang 
method 4 which includes the deletion heuristic, it is 
possikle to solve Bus after exploring only 15 nodes. Using 


method 1 with the addition heuristic requires develorirg 53 
nodes to solve Bus. Cn the other hand, the method using the 
deletion heuristic requires 4.24 seconds to solve the 
problem while the method using the addition heuristic 
reguires only 0.82 seconds to solve the problen. 

in order to illustrate the actual behavior Gf the arao- 
Tl een, the enumeration for Bus is shown below for tro 
different methods, method 2 and method 4. For these two 
methods, the enumeration trees are sufficiently small to be 
shown. The entire trees generated for Bus are displayed ir 
FIGUrG went ander tgure Seer Note that for both methcds, the 
optimal solution is found at the second node of the 
enumeration tree. Most of the running time of the aljorithma 
is spent proving optimality after the optimal solution is 


FOUnG: 
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Figure 5.1 Method 2 on Bus. 
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Method 4 on Bus. 


Paigure 5.2 
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VI. CONCLUSIONS AND RECOMMENDATIONS 

The Tranch and bound enumeration method using heuristi- 
cally obtained upper and lower bounds works weil cn scne 
epoblems and poorly cn others. Solution times are better 
than the times using the methods described bv Bausch on 
certain frobiems but other problems could not be solvel to 
optimality in a reasonable amount of time. The algorithm is 
largely dependent upcn the quality of bounds obtained, and 


in certain instances these bounds are not very Jjood. 


The greedy addition heuristic used here does not perforn 
as well as might be kcped and the deletion heuristic, which 
performs better, 1s too slow to use in most cases. Other 
addition heuristics should probably be tested which select 
that column j minimizing some function 


CC 2, ne), where g(c ,h_) is some function other than c sh. 
aves 7) SAS 


SIC Eds c¢ /10¢ (hay). MN pact ye VvaskoO andewilljams §! Ret. 51] 
J J 


have had some success selecting randomly from a number of 
such functions, albeit on randomly generated problems. Thkey 
atSOuptiluze a ~t=ort heuristic. Future research should 
examine the use of this and other exchange heuristics, 
Pace rch a CONnGIMCtLOM With “the addition heuristic 
Since it may be possible to significantiy improve upon the 
Solutions obtained without sacrificing much computational 
speed. 

The lower bound from the dual LP works quite well on 
some proklems and pocrly on others, HOtanly Iruck . Or 
course, the LP-based bound did not work well on Steinerl or 
Steiner2 since those problems were concocted so as to have 


very poor LF relaxaticns. The high speed of computation for 
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this lower Lound does allow rapid investigation of a large 
humber of nodes, however. In the Steiner problems, we 
expected that branch and bound enumeration using Xovac's 
lower bound might work better than the other lower bounds 
Since the value of the initial lower bounds were stronger 
than the other bounds as’ shown in Table 4. Unfortunately, 
the quality of the bound does not improve rapidly enough as 
the enumeration proceeds. Additional research is needed to 


generate ketter heuristic solution sets and lower bounds. 
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